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@ Moving picture decoding control system. 



@) In a nnoving picture decoding control system, 
when the amount of codes accumulated in a buffer 
memory is between first and second thresholds, the 
reading-out of the buffer memory is stopped and an 
image memory is repeatedly read out just after the I- 
picture or the P-picture of one frame is decoded 
until the amount of codes accumulated in the buffer 



memory exceeds the second threshold. When the 
amount of codes accumulated in the buffer memory 
is smaller than the first threshold, the reading-out of 
the buffer memory is stopped and the image mem- 
ory is repeatedly read out until the amount of codes 
accumulated in the buffer memory exceeds the sec- 
ond threshold. 
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Background of the Invention 

Field of the invention 

The present invention relates to a decoding 
system for decoding innage codes and having a 
code buffer memory and an image memory for 
matching in speed between a transmission rate and 
a decoding rate of encoded data. More specifically, 
the present invention relates to a system for con- 
trolling the code buffer memory and the image 
memory when the code amount becomes out of an 
ordinary capacity of the code buffer memory. 

Description of related art 

In encoding moving pictures, a code buffer 
memory has been used for matching in speed 
between a transmission rate and a decoding rate of 
encoded data. If the decoding rate of the encoded 
data is lower than the transmission rate of the 
encoded data, the encoded data is accumulated in 
the code buffer memory, and therefore, there is a 
fear that the code buffer memory overflows. If 
because of the overflow of the code buffer memory 
the encoded data is not stored in the code buffer 
memory and therefore is lost, it becomes impos- 
sible to continue the decoding. There is possibility 
that a substantial time is required until a normal 
operation is restored. Therefore, in the case that 
there is a fear that the code buffer memory over- 
flows, it is necessary to thin out the encoded data 
stored in the code buffer memory, in units which is 
hard to influence the decoding, for example, in 
units of one frame, so that an image stored in the 
image memory is outputted in place of a decoded 
image to be properly outputted. With this coun- 
termeasure, the encoded data stored in the code 
buffer memory can be reduced. 

On the other hand, if the decoding rate of the 
encoded data is higher than the transmission rate 
of the encoded data, the encoded data stored in 
the code buffer memory will reduce, and therefore, 
there is a fear that the code buffer memory under- 
flows. If the encoded data stored in the code buffer 
memory because of the underflow of the code 
buffer memory is lost, it becomes impossible to 
output a decoded image, so that the image be- 
comes broken. Therefore, in order to avoid a break 
of the image, when there is a fear that the code 
buffer memory underflows, it is necessary to stop 
supplying the encoded data stored in the code 
buffer memory to a decoding unit, and to output an 
image stored in the image memory of the decoding 
unit, in place of a decoded image to be properly 
outputted. With this arrangement, the encoded data 
stored in the code buffer memory can be in- 
creased. 



A conventional control of the code buffer mem- 
ory will be explained with reference to Figure 1 . 

The system shown in Figure 1 includes a code 
buffer memory controller 601, a code buffer mem- 

5 ory 602. a decoding unit 603. comparators 6041 
and 6042, and an image memory 605, coupled to 
each other as shown. A threshold Li is used in 
such a manner that, when an encoded data amount 
notice signal 6f becomes lower than this threshold 

w L1, it is discriminated that there is possibility of the 
underflow. This threshold LI is set to the amount of 
encoded data stored in the code buffer memory 
602 during a period of one frame. A threshold L4 is 
used in such a manner that, when the encoded 

75 data amount notice signal 6f becomes larger than 
this threshold L4, it is discriminated that there is 
possibility of the overflow. This threshold L4 is set 
to a value obtained by subtracting the amount of 
encoded data stored in the code buffer memory 

20 602 during a period of one frame, from a storage 
capacity of the code buffer memory 602. 

If the code buffer memory controller 601 de- 
tects on the basis of an output of the comparator 
6041 that the encoded data amount notice signal 6f 

25 becomes lower than the threshold LI. the code 
buffer memory controller 601 outputs a code buffer 
memory control signal 6a to the code buffer mem- 
ory 602 when the decoding unit 603 has completed 
the decoding of a frame under decoding, so that a 

30 reading-out of the code buffer memory 602 is stop- 
ped, and the encoded data is thinned out in units 
of for example one frame which is hard to influence 
on the decoding; In addition, the code buffer mem- 
ory controller 601 outputs a decoding control signal 

35 6b to the decoding unit 603 so as to stop the 
decoding operation of the decoding unit 603. The 
code buffer memory controller 601 outputs an im- 
age memory control signal 6e to the image mem- 
ory 605 so that a decoded image stored in the 

40 image memory is read out and outputted in place 
of a thinned-out image to be properly outputted. 

Thereafter, if the encoded data amount notice 
signal 6f becomes lower than the threshold L4, the 
code buffer memory controller 601 outputs the 

45 code buffer memory control signal 6a to the code 
buffer memory 602 so that the encoded data 
stored in the code buffer memory 602 is read out 
again. 1n addition, the code buffer memory control- 
ler 601 outputs the decoding control signal 6b to 

50 the decoding unit 603 so as to restart the decoding 
operation of the decoding unit 603. 

The above is the conventional control of the 
code buffei' memory and the image memory for 
preventing the overflow. 

55 In the MPEG (Moving Picture Experts Group) 

which is an internal standard for a moving picture 
compression system, a handling of the decoded 
image is different dependently upon a method for 
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predicting an image to be encoded. An image that 
was encoded within a frame without predicting on 
the basis of another image, is called an "l-picture". 
In the case of the l-picture, the image decoded by 
the decoding unit is stored in the image memory 
once, and thereafter, is outputted. An image that 
was encoded by predicting on the basis of a past 
image in a time sequence, is called an "P-picture". 
Similarly to the l-picture, In the case of the P- 
picture, the image decoded by the decoding unit is 
stored in the image memory once, and thereafter, 
is outputted. An image that was encoded by pre- 
dicting on the basis of a past image and a future 
image in a time sequence, is called an "B-picture". 
In the case of the P-picture, the image decoded by 
the decoding unit is outputted without being stored 
in the image memory. The decoded images stored 
in the image memory of the decoding unit are used 
for prediction at the time of decoding the "P- 
picture" and the "B-picture". 

As mentioned above, the decoded images 
stored in the image memory of the decoding unit 
includes only the "l-picture" and the "P-picture". 
and the "B-picture" is not stored in the image 
memory. For example, assume that the images 
outputted from the decoding unit are h, B2, B3, B4, 
B5, Bg, P7, Bs, in the time sequence as 

shown in (2) in Figure 4, and also assume that 
when the encoded data amount becomes lower 
than the threshold L1 in the course of the decoding 
of B3 so that there occurs a fear of the underflow, 
the supplying of the encoded data is stopped dur- 
ing a period of two frames. The images outputted 
from the decoding unit becomes I1 , B2, B3, I1 , li, 
B4. B5, Be, P7, Bs, in the time sequence as 
shown in ® in Figure 4. Namely, the image h is 
outputted next to the image B3 (point "i"). In other 
words, the decoded image is moved back by two 
frames. As a result, naturality is lost in the decoded 
image. In addition, the image Bs is outputted next 
to the image U (point "j"). Since We image Bs is 
separated from We image h by five frames, 
naturality is also lost at this point "j". 

Summary of the Invention 

Accordingly, it is an object of the present in- 
vention to provide a system for controlling a code 
buffer memory and an image memory in an en- 
coded image decoding system, which has over- 
come the above mentioned defect of the conven- 
tional one. 

Another object of the present invention is to 
provide a system for controlling a code buffer 
memory and an image memory in an encoded 
image decoding system, which can minimize un- 
natuality in a decoded image, by suppressing the 
above mentioned moving-back and jump of images 



in a time sequence, in an underflow preventing 
processing and an overflow preventing processing. 

The above and other objects of the present 
invention are achieved in accordance with the 

5 present invention by a moving picture decoding 
control system comprising a buffer memory for 
accumulating encoded images, a decoding unit for 
decoding an l-picture (in-frame encoded image), a 
P-picture (forward predicted encoded image) and a 

TO B-picture (bidirectional predicted encoded image) 
outputted from the buffer memory, an image mem- 
ory temporarily storing the l-picture and the P- 
picture decoded by the decoding unit, a compara- 
tor means for comparing the amount of codes 

75 accumulated in the buffer memory with first and 
second thresholds set from an external, the first 
threshold being smaller than the second threshold, 
the moving picture decoding control system being 
so configured that when the amount of codes accu- 

20 mutated in the buffer memory is between the first 
and second thresholds, the reading-out. of the buff- 
er memory is stopped and the image memory is 
repeatedly read out just after the l-picture or the P- 
picture of one frame is decoded until the amount of 

25 codes accumulated in the buffer memory exceeds 
the second threshold, and when the amount of 
codes accumulated in the buffer memory is smaller 
than the first threshold, the reading-out of the buff- 
er memory is stopped and the image memory is 

30 repeatedly read out until the amount of codes 
accumulated in the buffer memory exceeds the 
second threshold. 

According to another aspect of the present 
invention, there is provided a moving picture de- 

35 coding control system comprising a buffer memory 
for accumulating encoded images, a decoding unit 
for decoding an l-picture (in-frame encoded im- 
age), a P-picture (forward predicted encoded im- 
age) and a B-picture (bidirectional predicted en- 

40 coded image) outputted from the buffer memory, 
an image memory temporarily storing the l-picture 
and the P-picture decoded by the decoding unit, a 
comparator means for comparing the amount of 
codes accumulated in the buffer memory with first 

45 and second thresholds set from an external, the 
first threshold being smaller than the second 
threshold, the moving picture decoding control sys- 
tem being so configured that when the amount of 
codes accumulated in the buffer memory is be- 

50 tween the first and second thresholds, all codes 
read from the buffer memory are skipped and an 
image is outputted from the image memory in 
place of a frame corresponding the codes skipped, 
just after the l-picture or the P-picture of one frame 

55 is decoded until the amount of codes accumulated 
in the buffer memory becomes lower than the first 
threshold, and when the amount of codes accu- 
mulated in the buffer memory is larger than the 
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second threshold, all codes read from the buffer 
memory are skipped and an image is outputted 
from the image memory in place of a frame cor- 
responding the codes skipped until the amount of 
codes accumulated in the buffer memory becomes 
lower than the first threshold. 

According to still another aspect of the present 
invention, there is provided a moving picture de- 
coding control system comprising a buffer memory 
for accumulating encoded images, a decoding unit 
for decoding an l-picture (in-frame encoded im- 
age), a P-picture (forward predicted encoded im- 
age) and a B-picture (bidirectional predicted en- 
coded image) outputted from the buffer memory, 
an image memory temporarily storing the l-picture 
and the P-picture decoded by the decoding unit, a 
comparator means for comparing the amount of 
codes accumulated in the buffer memory with first 
and second thresholds set from an external, the 
first threshold being smaller than the second 
threshold, the moving picture decoding control sys- 
tem being so configured that when the amount of 
codes accumulated in the buffer memory is be- 
tween the first and second thresholds, only codes 
of the B-picture read from the buffer memory are 
skipped and an image is outputted from the image 
memory in place of a frame corresponding the 
codes skipped, just after the l-picture or the P- 
picture of one frame is decoded until the amount of 
codes accumulated in the butter memory becomes 
lower than the first threshold, and when the amount 
of codes accumulated in the buffer memory Is 
larger than the second threshold, all codes read 
from the buffer memory are skipped and an image 
is outputted from the image memory in place of a 
frame corresponding the codes skipped until the 
amount of codes accumulated in the buffer mem- 
ory becomes between the first and second thresh- 
olds. 

The above and other objects, features and ad- 
vantages of the present invention will be apparent 
from the following description of preferred embodi- 
ments of the invention with reference to the accom- 
panying drawings. 

Brief Description of the Drawings 

Figure 1 is a block diagram of a conventional 
moving picture decoding control system; 
Figure 2 is a block diagram of a first embodi- 
ment of the moving picture decoding control 
system in accordance with the present inven- 
tion; 

Figure 3 is a flow chart illustrating an operation 
of the first embodiment; 

Figure 4 illustrates examples of images output- 
ted from the decoding unit; 



Figure 5 is a block diagram of a second em- 
bodiment of the moving picture decoding control 
system in accordance with the present inven- 
tion; and 

5 Figure 6 is a flow chart illustrating an operation 

of the second embodiment. 

Description of the Preferred embodiments 

10 Referring to Figure 2, there is shown a block 

diagram of a first embodiment of the moving pic- 
ture decoding control system in accordance with 
the present invention. 

In the shown embodiment, encoded data sup- 

75 plied through a transmission channel (not shown) is 
supplied to a code buffer memory 102, which in 
turn has an output connected to a decoding unit 
103. A decoded image outputted from the decod- 
ing unit 103 is supplied to an image memory 105 

20 and also outputted to an external. In addition, a 
decoded image read from the image memory 105 
is supplied to the decoding unit 103. The decoding 
unit 103 outputs an encoded data request signal 1c 
and a picture type notice signal Id to a code buffer 

25 memory controller 101 . 

The code buffer memory controller 101 outputs 
a code buffer memory control signal la to the code 
buffer memory 102, and also outputs a decoding 
control signal 1b to the decoding unit 103. Further- 

30 more, the code buffer memory controller 101 out- 
puts an image memory control signal 1e to the 
image memory 105, and supplies an encoded data 
amount notice signal If to one input of each of four 
comparators 1041, 1042. 1043 and 1044. 

35 A first threshold LI is supplied to the other 

input of the comparator 1041, and a second thresh- 
old L2 is supplied to the other input of the com- 
parator 1042. A third threshold L3 is supplied to the 
other input of the comparator 1043, and a fourth 

40 threshold L4 is supplied to the other input of the 
comparator 1044. An output of each of the four 
comparators 1041, 1042, 1043 and 1044 is sup- 
plied to the code buffer memory controller 101. 
The first threshold LI is set to the amount of 

45 encoded data stored in the code buffer memory 
102 during a period of one frame. The second 
threshold L2 is set to the amount of encoded data 
stored in the code buffer memory 102 during a 
period of two frames. The third threshold L3 is set 

50 to a value obtained by subtracting the amount of 
encoded data stored in the code buffer memory 
102 during a period of two frames, from a maxi- 
mum storage capacity of the code buffer memory 
102. The fourth threshold L4 is set to a value 

55 obtained by subtracting the amount of encoded 
data stored in the code buffer memory 102 during 
a period of one frame, from the storage capacity of 
the code buffer memory 102. 
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Now, operation of the above mentioned first 
embodiment will be explained witli reference to 
Figure 3 which is a flow chart illustrating an opera- 
tion of the first embodiment. 

In a normal operation in which there is neither 
a fear of overflow nor a fear of underflow, the code 
buffer memory controller 101 controls the code 
buffer memory 102 by the code buffer memory 
control signal la so that the encoded data supplied 
through the transmission channel is written into the 
code buffer memory 102. On the other hand, the 
decoding unit 103 outputs the encoded data re- 
quest signal 1c to the code buffer memory control- 
ler 101. and in response to the encoded data 
request signal 1c. the code buffer memory control- 
ler 101 controls the code buffer memory 102 by 
the code buffer memory control signal la so as to 
cause the code buffer memory 102 to output the 
encoded data to the decoding unit 103. The decod- 
ing unit 103 having received the encoded data, 
decodes the encoded data so as to generate a 
decoded image, and also generates the picture 
type notice signal Id. In the code buffer memory 
controller 101 has known from picture type notice 
signal Id from the decoding unit 103 that the 
decoded image is either the l-picture or the P- 
picture, the code buffer memory controller 101 
controls the image memory by the image memory 
control signal 1e so as to write the decoded image 
into the image memory 105. 

If it is designed that in this normal operation, 
the amount of the encoded data accumulated in the 
code buffer memory 102 varies between the sec- 
ond threshold L2 and the third threshold L3, neither 
the underflow nor the overflow occurs. 

However, when the encoded data has changed 
in the transmission channel or when an error oc- 
curs in the decoding unit, or alternatively, when an 
actual difference between the transmission rate and 
the decoding rate is larger than a designed value, 
there is possibility that the underflow or the over- 
flow occurs. For complying with such a situation, 
the code buffer memory and the image memory 
are controlled as shown in the flow chart of Figure 
3. 

In the normal operation, the code buffer mem- 
ory controller 101 ceaselessly monitors on the ba- 
sis of the outputs of the four comparators 1041, 
1042, 1043 and 1044 the large-and-small relation 
between the encoded data amount notice signal If 
and the thresholds LI, L2, L3 and L4. while control- 
ling the normal decoding operation. If the situation 
becomes under the following case "a" or "b", the 
code buffer memory controller 101 changes to the 
normal operation to an underflow preventing pro- 
cessing or art overflow preventing processing. 
Case "a" 



When the encoded data amount police signal 
If is not larger than the threshold L2. the following 
underflow preventing processing is carried out. 

5 Step 1 

The completion of the decoding operation of a 
frame being decoded in the decoding unit 103 is 
waited, 

10 

Step 2 

If the encoded data amount notice signal 1f is 
not larger than the threshold LI. the operation goes 
75 to Step 4. Otherwise, the operation goes to Step 3. 

Step 3 

The code buffer memory controller 101 dis- 
20 criminates the picture type notice signal Id output- 
ted from the decoding unit 103. and if the decoding 
completed frame is either the l-picture or the P- 
picture, the operation goes to Step 4. Otherwise, 
the operation goes to Step 8. 

25 

Step 4 

The code buffer memory controller 101 in- 
structs the code buffer memory 102 by the code 
30 buffer memory control signal la so as to cause the 
code buffer memory 102 to stop supplying the 
encoded data to the decoding unit 103. 

Step 5 

35 

In place of the image which cannot be output- 
ted from the decoding unit 103 because the de- 
coded data is not supplied to the decoding unit 
103, the code buffer memory controller 101 con- 

40 trols the decoding unit 103 by the decoding control 
signal lb so as to cause the decoding unit 103 to 
stop its decoding operation, and also controls the 
image memory 105 by the image memory control 
signal 1e so as to cause an image stored in the 

45 image memory 105 to be read out. 

Step 6 

If the encoded data amount notice signal 1f is 
50 not larger than the- threshold L2, the operation goes 
to Step 5. Otherwise, the operation goes to Step 7. 

Step 7 

55 In order to restart the supply of the encoded 

data to the decoding unit 103. the code buffer 
memory controller 101 controls the code buffer 
memory 102 by the code buffer memory control 
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signal la to cause the code buffer memory 102 to 
restart the reading-out operation. Then, the opera- 
tion goes to Step 9. 

Step 8 

In order to decode a next frame, the code 
buffer memory controller 101 controls the code 
buffer memory 102 by the code buffer memory 
control signal la to cause the code buffer memory 
102 to execute the reading-out operation. In addi- 
tion, the code buffer memory controller 101 con- 
trols the decoding unit 103 by the decoding control 
signal lb so as to cause the decoding unit 103 to 
decode the encoded data received from the code 
buffer memory 102. Then, the operation goes to 
the step 1 . 

Step 9 

The code buffer memory controller 101 con- 
trols the decoding unit 103 by the decoding control 
signal lb so as to cause the decoding unit 103 to 
restore the decoding operation. As a result, the 
operation returns to the normal decoding operation. 

Case "b" 

if the encoded data amount notice signal If is 
not smaller than the threshold L3. the following 
overflow preventing processing is performed. 

Step 1 

The completion of the decoding operation of a 
frame being decoded in the decoding unit 103 is 
waited. 

Step 2 

If the encoded data amount notice signal If is 
not smaller than the threshold L4, the operation 
goes to Step 4. Otherwise, the operation goes to 
Step 3. 

Step 3 

The code buffer memory controller 101 dis- 
criminates the picture type notice signal Id output- 
ted from the decoding unit 103, and if the decoding 
completed frame is either the l-picture or the P- 
picture, the operation goes to Step 4. Otherwise, 
the operation goes to Step 9. 

Step 4 

The code buffer memory controller 101 in- 
structs the code buffer memory 102 by the code 
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buffer memory control signal la so as to cause the 
code buffer memory 102 to stop supplying the 
encoded data to the decoding unit 103. 

5 Step 5 

The code buffer memory controller 101 con- 
trols the code buffer memory 102 by the code 
buffer memory control signal la to cause to thin 
70 out (or erase) the encoded data of one frame from 
the code buffer memory 102. 

Step 6 

75 The code buffer memory controller 101 con- 

trols the decoding unit 103 by the decoding control 
signal lb so as to cause the decoding unit 103 to 
stop its decoding operation. In addition, in place of 
the image which cannot be outputted from the 

20 decoding unit 103 because the decoded data is 
thinned out in the Step 5, the code buffer memory 
controller 101 controls the image memory 105 by 
the image memory control signal 1e so as to cause 
an image stored in the image memory 105 to be 

25 read out. 

Step 7 

If the encoded data amount notice signal If is 
30 not smaller than the threshold L3, the operation 
goes to Step 5. Otherwise, the operation goes to 
Step 8. 

Step 8 

35 

In order to restart the supply of the encoded 
data to the decoding unit 103, the code buffer 
memory controller 101 controls the code buffer 
memory 102 by the code buffer memory control 
40 signal la to cause the code buffer memory 102 to 
restart the reading-out operation. Then, the opera- 
tion goes to Step 10. 

Step 9 

45 

In order to decode a next frame, the code 
buffer memory controller 101 controls the code 
buffer memory 1 02 by the code buffer memory 
control signal 1 a to cause the code buffer memory 

50 102 to execute the reading-out operation. In addi- 
tion, the code buffer memory controller 101 con- 
trols the decoding unit 103 by the decoding control 
signal lb so as to cause the decoding unit 103 to 
decode the encoded data received from the code 

55 buffer memory 102. Then, the operation goes to 
the step 1 . 
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Step 10 

The code buffer memory controller 101 con- 
trols the decoding unit 103 by the decoding control 
signal 1b so as to cause the decoding unit 103 to 
restore the decoding operation. As a result, the 
operation returns to the normal decoding operation. 

Now, examples of the outputted image in this 
embodiment will be explained with reference to 
Figure 4 Illustrating examples of images outputted 
from the decoding unit. 

In Figure 4, the image sequence ® shows the 
transmitted order of the encoded data In designates 
an l-picture outputted as a decoded image at an 
(n)th place. Pn shows a P-plcture outputted as a 
decoded image at an (n)th place. In addition, Bp 
Indicates a B-picture outputted as a decoded im- 
age at an (n)th place. In addition, the image se- 
quence © shows the order of the decoded Images 
outputted from the decoding unit when it is unnec- 
essary to perform either the underflow preventing 
processing or the overflow preventing processing. 

Now, assume that when the decoding operation 
is continuously performed, the encoded data 
amount accumulated in the code buffer memory 
becomes lower than the threshold L2 at a time "a", 
and also becomes lower than the threshold LI at a 
time "b". Also assume that when the underflow 
preventing processing is executed in this embodi- 
ment and in accordance with the conventional ex- 
ample explained hereinbefore, the encoded data 
amount becomes higher than the threshold L2 dur- 
ing a period of two frames in accordance with this 
embodiment, and becomes higher than the thresh- 
old LI during a period of two frames in accordance 
with the conventional example. Thereafter, the un- 
derflow preventing processing becomes unneces- 
sary. Under these condition, the sequence of im- 
ages outputted from the decoding unit In this em- 
bodiment becomes as shown in @ of Figure 4, 
and the sequence of images outputted from the 
decoding unit in the conventional example be- 
comes as shown in @ of Figure 4. 

In this embodiment, the encoded data amount 
becomes lower than the threshold L2 at the time 
"a", and when the decoding of Pi is completed, 
the supply of the encoded data is stopped. In place 
of the image which cannot be decoded because 
the encoded data is not supplied to the decoding 
unit, the image h stored in the image memory 105 
is read out and outputted. Since the encoded data 
amount becomes higher than the threshold L2 dur- 
ing the period of two frames, the image h is 
outputted two times. Thereafter, the operation re- 
turns to the normal decoding operation, so that the 
images are outputted in the order shown in (3) of 
Figure 4. 



In the conventional example, on the other hand, 
the encoded data amount becomes lower than the 
threshold LI at the time "b", and when the decod- 
ing of B3 is completed, the supply of the encoded 

5 data is stopped. In place of the image which cannot 
be decoded because the encoded data is not sup- 
plied to the decoding unit, the image li stored In 
the image memory 105 is read out arid outputted. 
Since the encoded data amount becomes higher 

10 than the threshold LI during the period of two 
frames, the Image h is outputted two times. There- 
after, the operation returns to the normal decoding 
operation, so that the images are outputted in the 
order shown in @ of Figure 4. 

75 Furthermore, assume that when the decoding 

operation is continuously performed, the encoded 
data amount accumulated in the code buffer mem- 
ory becomes higher than the threshold L3 at a time 
"a", and also becomes higher than the threshold 

20 L4 at a time "b'V Also assume that when the 
overflow preventing processing is executed in this 
embodiment and in accordance with the conven- 
tional example explained hereinbefore, the encoded 
data amount becomes lower than the threshold L3 

25 during a period of two frames in accordance with 
this embodiment, and becomes lower than the 
threshold L4 during a period of two frames in 
accordance with the conventional example. There- 
after, the overflow preventing processing becomes 

30 unnecessary. Under these condition, the sequence 
of images outputted from the decoding unit in this 
embodiment becomes as shown in © of Figure 4, 
and the sequence of images outputted from the 
decoding unit in the conventional example be- 

35 comes as shown in (g) of Figure 4. 

In this embodiment, the encoded data amount 
becomes higher than the threshold L3 at the time 
"a", and when the decoding of P7 is completed, 
the supply of the encoded data is stopped and the 

40 encoded data for the next image B2 is thinned out 
or deleted. In place of the image B2 which cannot 
be decoded because it is thinned out or deleted, 
the Image h stored in the image memory 105 is 
read out and outputted. Since the encoded data 

46 amount becomes lower than the threshold L3 dur- 
ing the period of two frames, the Image B3 is also 
thinned out or deleted. Therefore, the image h is 
outputted two times. Thereafter, the operation re- 
turns to the normal decoding operation, so that the 

50 images are outputted in the order shown in (§) of 
Figure 4. 

In the conventional example, on the other hand, 
the encoded data amount becomes higher than the 
threshold L4 at the time "b", and when the decod- 
55 ing of B3 is completed, the supply of the encoded 
data is stopped, and the encoded data for the next 
image B* is thinned out or deleted. In place of the 
image B4 which cannot be decoded because it is 
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thinned out or deleted, the image h stored in the 
image memory 105 is read out and outputted. 
Since the encoded data amount becomes lower 
than the threshold L4 during the period of two 
framed, the image B5 is also thinned out or de- 
leted. Therefore, the image h is outputted two 
times. Thereafter, the operation returns to the nor- 
mal decoding operation, so that the images are 
outputted in the order shown in (§) of Figure 4. 

Referring to Figure 5, there is shown a block 
diagram of a second embodiment of the moving 
picture decoding control system in accordance with 
the present invention. The second embodiment 
shown in Figure 5 is a modification of the first 
embodiment shown in Figure 2 in which the means 
for the underflow preventing processing is removed 
so that only the overflow preventing processing is 
executed 

In the second embodiment, encoded data sup- 
plied through a transmission channel (not shown) is 
supplied to a code buffer memory 402, which in 
turn has an output connected to a decoding unit 
403. A decoded image outputted from the decod- 
ing unit 403 is supplied to an image memory 405 
and also outputted to an external. In addition, a 
decoded image read from the image memory 405 
is supplied to the decoding unit 403. The decoding 
unit 403 outputs an encoded data request signal 4c 
and a picture type notice signal 4d to a code buffer 
memory controller 401. 

The code buffer memory controller 401 outputs 
a code buffer memory control signal 4a to the code 
buffer memory 402, and also outputs a decoding 
control signal 4b to the decoding unit 403. Further- 
more, the code buffer memory controller 401 out- 
puts an image memory control signal 4e to the 
image memory 405. and supplies an encoded data 
amount notice signal 4f to one input of each of four 
connparators 4041 and 4042. 

A threshold* L3 is supplied to the other input of 
the comparator 4041, and another threshold L4 is 
supplied to the other input of the comparator 4042. 
An output of each of the four comparators 4041 
and 4042 is supplied to the code buffer memory 
controller 401. The threshold L3 is set to a value 
obtained by subtracting the amount of encoded 
data stored in the code buffer memory 402 during 
a period of two frames, from a maximum storage 
capacity of the code buffer memory 402. The 
threshold L4 is set to a value obtained by subtrac- 
ting the amount of encoded data stored in the code 
buffer memory 402 during a period of one frame, 
from the storage capacity of the code buffer mem- 
ory 402. 

Now, operation of the above mentioned second 
embodiment will be explained with reference to 
Figure 6 which is a flow chart illustrating an opera- 
tion of the second embodiment. 
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In a normal operation, the code buffer memory 
controller 401 controls the code buffer memory 402 
by the code buffer memory control signal 4a so 
that the encoded data supplied through the trans- 

5 mission channel is written into the code buffer 
memory 402. On the other hand, the decoding unit 
403 outputs the encoded data request signal 4c to 
the code buffer memory controller 401, and in 
response to the encoded data request signal 4c, 

70 the code buffer memory controller 401 controls the 
code buffer memory 402 by the code buffer mem- 
ory control signal 4a so as to cause the code buffer 
memory 402 to output the encoded data to the 
decoding unit 403. The decoding unit 403 having 

75 received the encoded data, decodes the encoded 
data so as to generate a decoded image, and also 
generates the picture type notice signal 4d. If the 
code buffer memory controller 401 has known from 
picture type notice signal 4d from the decoding 

20 unit 403 that the decoded image is either the I- 
picture or the P-picture. the code buffer memory 
controller 401 controls the image memory 405 by 
the image memory control signal 4e so as to write 
the decoded image into the image memory 405. 

25 If it is designed that in this normal operation, 

the amount of the encoded data accumulated in the 
code buffer memory 402 is maintained to be under 
the threshold L3, the overflow does not occur. 

However, when the encoded data has changed 

30 in the transmission channel or when an error oc- 
curs in the decoding unit, or alternatively, when the 
transmission rate is extremely higher than the de- 
coding rate, there is possibility that the overflow 
occurs. For complying with such a situation, the 

35 code buffer memory and the image memory are 
controlled as shown in the flow chart of Figure 6. 

In the normal operation, the code buffer mem- 
ory controller 401 ceaselessly monitors on the ba- 
sis of the outputs of the comparators 4041 and 

40 4042 the large-and-smali relation between the en- 
coded data amount notice signal 4f and the thresh- 
olds L3 and L4, while controlling the normal decod- 
ing operation. If the situation becomes under the 
following case "a", the code buffer memory con- 

45 troller 401 changes to the normal operation to an 
overflow preventing processing. Case "a" 

If the encoded data amount notice signal 4f is 
not smaller than the threshold L3, the following 
overflow preventing processing is performed. 

50 

Step 1 

The completion of the decoding operation of a 
frame being decoded in the decoding unit 403 is 
55 waited. 
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Step 2 

If the encoded data amount notice signal 4f is 
not smaller than the threshold L4, the operation 
goes to Step 4. Otherwise, the operation goes to 
Step 3. 

Step 3 

The code buffer memory controller 401 dis- 
criminates the picture type notice signal 4d output- 
ted from the decoding unit 403, and if the decoding 
completed frame is either the l-picture or the P- 
picture, the operation goes to Step 4. Otherwise, 
the operation goes to Step 9. 

Step 4 

The code buffer memory controller 401 in- 
structs the code buffer memory 402 by the code 
buffer memory control signal 4a so as to cause the 
code, buffer memory 402 to stop supplying the 
encoded data to the decoding unit 403. 

Step 5 

The code buffer memory controller 401 con- 
trols the code buffer memory 402 by the code 
buffer memory control signal 4a to cause to thin 
out (or erase) the encoded data of one frame from 
the code buffer memory 402. 

Step 6 

The code buffer memory controller 401 con- 
trols the decoding unit 403 by the decoding control 
signal 4b so as to cause the decoding unit 403 to 
stop its decoding operation. In addition, in place of 
the image which cannot be outputted from the 
decoding unit 403 because the decoded data is not 
supplied, the code buffer memory controller 401 
controls the image memory 405 by the image 
memory control signal 4e so as to cause an image 
stored in the image memory 405 to be read out. 

Step 7 

If the encoded data amount notice signal 4f is 
not smaller than the threshold L3. the operation 
goes to Step 5. Otherwise, the operation goes to 
Step 8. 

Step 8 

In order to restart the supply of the encoded 
data to the decoding unit 403, the code buffer 
memory controller 401 controls the code buffer 
memory 402 by the code buffer memory control 
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signal 4a to cause the code buffer memory 402 to 
restart the reading-out operation Then, the opera- 
tion goes to Step 10. 

5 Step 9 

In order to decode a next frame, the code 
buffer memory controller 401 controls the code 
buffer memory 402 by the code buffer memory 

10 control signal 4a to cause the code buffer memory 
402 to execute the reading-out operation. In addi- 
tion, the code buffer memory controller 401 con- 
trols the decoding unit 403 by the decoding control 
signal 4b so as to cause the decoding unit 403 to 

75 decode the encoded data received from the code 
buffer memory 402. Then, the operation goes to 
the step 1 . 

Step 10 

20 

The code buffer memory controller 401 con- 
trols the decoding unit 403 by the decoding control 
signal 4b so as to cause the decoding unit 403 to 
restore the decoding operation. As a result, the 

25 operation returns to the norma! decoding operation. 

As will be seen from the above, the sequence 
of images outputted from the second embodiment 
is the same as that of images outputted when the 
overflow preventing processing is executed in the 

30 first embodiment. 

As apparent from above, the moving picture 
decoding control systenri in accordance with the 
present invention has a plurality of thresholds for 
each of the underflow preventing processing and 

35 the overflow preventing processing,- and is so con- 
figured that different processings are performed 
dependently upon the decoded data amount com- 
pared with the thresholds. 

In the underflow preventing processing, when 

40 the amount of encoded data accumulated in the 
code buffer memory is still relatively much (name- 
ly, It becomes lower than the threshold L2), the 
supply of the encoded data to the decoding unit is 
stopped so that the decoding operation is tem- 

45 porarily stopped. However, since the decoding Is 
stopped just after the decoding of the l-picture or 
the P-picture is completed, the image (li) just 
before the decoding operation Is stopped can be- 
come the same as the Image (h) outputted during 

50 a period of time in which the decoding operation is 
being stopped, as shown at the timing "c" In @ of 
Figure 4. Accordingly, the Image Is never moved 
back in time sequence. In addition, no skip exists 
between the image (h ) outputted during a period of 

55 time in which the decoding operation is being stop- 
ped, and the image (B2) outputted after the decod- 
ing operation is restarted, at the timing "d" in @ of 
Figure 4, 
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Accordingly, unnaturality in innage is mini- 
mized, in comparison with the conventional under- 
flow preventing processing (the image is moved 
back in time sequence by two frames at the timing 
"e" in @ of Figure 4, and the image is skipped by 
three frames at the timing "f" in ® of Figure 4). 

Furthermore, after the encoded data amount 
becomes lower than the threshold l_2, if the en- 
coded data amount becomes lower than the thresh- 
old LI when the l-picture or the P-picture is waited, 
the decoding of the next image is unconditionally 
stopped similarly to the conventional example. 
Therefore, the underflow will never occur. 

In the overflow preventing processing, when 
the amount of encoded data accumulated in the 
code buffer memory is still relatively much (name- 
ly, it becomes higher than the threshold L3), the 
decoding operation is temporarily stopped and the 
encoded data is thinned out. However, since the 
decoding is stopped just after the decoding of the 
l-picture or the P-picture is completed, the image 
(h) just before the decoding operation is stopped 
can become the same as the image {\^) outputted 
during a period of time in which the decoding 
operation is being stopped, as shown at the timing 
"g" in ® of Figure 4. Accordingly, the image is 
never moved back in time sequence. In addition, 
the skip between the image (li) outputted during a 
period of time in which the decoding operation is 
being stopped, and the image (B4) outputted after 
the decoding operation is restarted, is relatively 
small at the timing "h" in ® of Figure 4. 

Accordingly, unnaturality in image is mini- 
mized, in comparison with the conventional over- 
flow preventing processing (the image is moved 
back in time sequence by two frames at the timing 
"i" in ® of Figure 4, and the image is skipped by 
five frames at the timing "j" in (g) of Figure 4). 

Furthermore, after the encoded data amount 
becomes higher than the threshold L3, if the en- 
coded data amount becomes higher than the 
threshold L4 when the l-picture or the P-picture is 
waited, the decoding of the next image is uncondi- 
tionally stopped and the decoded data is imme- 
diately thinned out, similarly to the conventional 
example. Therefore, the overflow will never occur. 

Alternatively, if only the B-pictures are thinned 
out or deleted when the amount of encoded data 
accumulated in the code buffer memory is between 
the threshold L3 and the threshold L4, the l-picture 
and the P-picture are decoded in the course of the 
overflow preventing processing, and as soon as the 
decoded image is outputted, the decoded image is 
written into the image memory. In this case, un- 
naturality in image is further minimized. In addition, 
it is possible to properly decode the predicted 
encoded image (P-picture and B-picture) to be 
decoded just after the overflow preventing process- 



ing is completed. 

As seen from the above, the system in accor- 
dance with the present invention for controlling the 
code buffer memory and the image memory can 

5 minimize the moving-back of the image in the time 
sequence and the skip of the image in the time 
sequence, the underflow and/or overflow preventing 
processing, with the result that unnaturality in im- 
age can be minimized. In addition, even if the code 

10 buffer memory does not have a sufficient storage 
capacity, and therefore, the overflow preventing 
processing is frequently carried out, it is possible 
to reduce the unnaturality of the decoded image. 
The invention has thus been shown and de- 

76 scribed with reference to. the specific embodi- 
ments. However, it should be noted that the 
present invention is in no way limited to the details 
of the illustrated structures but changes and modi- 
fications may be made within the scope of the 

20 appended claims. 

Claims 

1. A moving picture decoding control system 
25 comprising a buffer memory for accumulating 

encoded Images, a decoding unit for decoding 
an l-picture (in-frame encoded image), a P- 
picture (forward predicted encoded image) and 
a B-picture (bidirectional predicted encoded 
30 image) outputted from the buffer memory, an 

image memory temporarily storing the l-picture 
and the P-picture decoded by the decoding 
unit, a comparator means for comparing the 
amount of codes accumulated in the buffer 
35 memory with first and second thresholds set 

from an external, the first threshold being 
smaller than the second threshold, the moving 
picture decoding control system being so con- 
figured that when the amount of codes accu- 
se mulated in the buffer memory is between the 
first and second thresholds, the reading-out of 
the buffer memory is stopped and the image 
memory is repeatedly read out just after the I- 
picture or the P-picture of one frame is de- 
45 coded until the amount of codes accumulated 
in the buffer memory exceeds the second 
threshold, and when the amount of codes ac- 
cumulated in the buffer memory is smaller 
than the first threshold, the reading-out of the 
50 buffer memory is stopped and the image 
memory is repeatedly read out until the 
amount of codes accumulated in the buffer 
memory exceeds the second threshold. 

55 2. A moving picture decoding control system 
comprising a buffer memory for accumulating 
encoded images, a decoding unit for decoding 
an l-picture (in-frame encoded image), a P- 
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picture (forward predicted encoded image) and 
a B-picture (bidirectional predicted encoded 
image) outputted from the buffer memory, an 
image memory temporarily storing the l-picture 
and the P-picture decoded by the decoding 5 
unit, a comparator means for comparing the 
amount. of codes accumulated in the buffer 
memory with first and second thresholds set 
from an external, the first threshold being 
smaller than the second threshold, the moving w 
picture decoding control system being so con- 
figured that when the amount of codes accu- 
mulated in the buffer memory is between the 
first and second thresholds, all codes read 
from the buffer memory are skipped and an is 
image is outputted from the image memory in 
place of a frame corresponding the codes skip- 
ped, just after the l-picture or the P-picture of 
one frame is decoded until the amount of 
codes accumulated in the buffer memory be- 20 
comes lower than the first threshold, and when 
the amount of codes accumulated in the buffer 
memory is larger than the second threshold, all 
codes read from the buffer memory are skip- 
ped and an image is outputted from the image 25 
memory in place of a frame corresponding the 
codes skipped until the amount of codes accu- 
mulated in the buffer memory becomes lower 
than the first threshold. 

30 

A moving picture decoding control system 
comprising a buffer memory for accumulating 
encoded images, a decoding unit for decoding 
an l-picture (in-frame encoded image), a P- 
plcture (forward predicted encoded image) and 35 
a B-picture (bidirectional predicted encoded 
image) outputted from the buffer memory, an 
image memory temporarily storing the l-picture 
and the P-picture decoded by the decoding 
unit, a comparator means for comparing the 40 
amount of codes accumulated in the buffer 
memory with first and second thresholds set 
from an external, the first threshold being 
smaller than the second threshold, the moving 
picture decoding control system being so con- 45 
figured that when the amount of codes accu- 
mulated in the buffer memory is between the 
first and second thresholds, only codes of the 
B-picture read from the buffer memory are 
skipped and an image is outputted from the 50 
image memory in place of a frame correspond- 
ing the codes skipped, just after the l-picture 
or the P-picture of one frame is decoded until 
the amount of codes accumulated in the buffer 
memory becomes lower than the first thresh- 55 
old, and when the amount of codes accu- 
mulated in the buffer memory is larger than 
the second threshold, all codes read from the 



buffer memory are skipped and an image is 
outputted from the. image memory in place of 
a frame corresponding the codes skipped until 
the amount of codes accumulated in the buffer 
memory becomes between the first and sec- 
ond thresholds. 
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@ Moving picture decoding control system. 



@ In a moving picture decoding control system, 
when the amount of codes accumulated in a buffer 
memory is between first and second thresholds, the 
reading-out of the buffer memory is stopped and an 
image memory is repeatedly read out just after the I- 
picture or the P-picture of one frame is decoded 
until the amount of codes accumulated in the buffer 



memory exceeds the second threshold. When the 
amount of codes accumulated in the buffer memory 
is smaller than the first threshold, the reading-out of 
the buffer memory is stopped and the image mem- 
ory is repeatedly read out until the amount of codes 
accumulated in the buffer memory exceeds the sec- 
ond threshold. 
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